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Abstract 

Noncooperative game theory provides a normative framework for analyzing strategic inter- 
actions. However, for the toolbox to be operational, the solutions it defines will have to be 
computed. In this paper, we provide a single reduction that 1) demonstrates jVP-hardness 
of determining whether Nash equilibria with certain natural properties exist, and 2) demon- 
strates the ^P-hardness of counting Nash equilibria (or connected sets of Nash equilibria). 
We also show that 3) determining whether a pure-strategy Bayes-Nash equilibrium exists 
is A^P-hard, and that 4) determining whether a pure-strategy Nash equilibrium exists in 
a stochastic (Markov) game is VSVACS-hsnd even if the game is invisible (this remains 
ATP-hard if the game is finite). All of our hardness results hold even if there are only two 
players and the game is symmetric. 
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1 Introduction 



Noncooperative game theory provides a normative framework for analyzing strategic inter- 
actions. However, for the toolbox to be operational, the solutions it defines will have to be 
computed f47 |. There has been growing interest in the computational complexity of natu- 



ral questions in game theory. Starting at least as early as the 1970s, complexity theorists 
have focused on the complexity of playing particular highly structured games (usually board 
games, such as chess or Go PE|] , but also games such as Geography or QSAT [0). These 
games tend to be alternating-move zero-sum games with enormous state spaces, which can 
nevertheless be concisely represented due to the simple rules governing the transition be- 
tween states. As a result, effort on finding results for general classes of games has often 
focused on complex languages in which such structured games can be concisely represented. 

Real-world strategic settings are generally not nearly as structured, nor do they gener- 
ally possess the other properties (most notably, zero-sumness) of board games and the like. 
Algorithms for analyzing this more general class of games strategically are a necessary com- 
ponent of sophisticated agents that are to play such games. Additionally, they are needed 
by mechanism designers who have (some) control over the rules of the game and would like 
the outcome of the game to have certain properties, such as maximum social welfare. 

Noncooperative game theory provides languages for representing large classes of strategic 
settings, as well as sophisticated notions of what it means to " solve" such games. The best 



known solution concept is that of Nash equilibrium [31], where the players' strategies are 



such that no individual player can derive any benefit from deviating from its strategy. The 
question of how complex it is to construct such an equilibrium has been dubbed "a most 
fundamental computational problem whose complexity is wide open" and "together with 
factoring, [...] the most important concrete open question on the boundary of V today" [[38 



While this question remains open, important concrete advances have been made in de- 
termining the complexity of related questions. For example, 2-person zero-sum games can 
be solved using linear programming |27| in polynomial time. As another example, deter- 
mining the existence of a joint strategy where each player gets expected payoff at least k in 
a team game in extensive form is A/'P-complete (as the number of players grows) [fj. As 
yet another example, in 2-player general-sum normal form games, determining the existence 
of Nash equilibria with certain properties is A/"P-hard |jl3| . Finally, the complexity of best- 
responding and of guaranteeing payoffs in repeated and sequential games has been studied 
in [|,|Oi0, 0. 



In this paper we provide new complexity results on questions related to Nash equilibria. 
In Section we provide a single reduction which significantly improves on many of Gilboa 
and Zemel's results on determining the existence of Nash equilibria with certain properties. 
In Section |3|, we use the same reduction to show that counting the number of Nash equilibria 
(or connected sets of Nash equilibria) is #P-hard. In Section [| we show that determining 
whether a pure-strategy Bayes-Nash equilibrium exists is A/'P-hard. Finally, in Section |5] 
we show that determining whether a pure-strategy Nash equilibrium exists in a stochastic 
(Markov) game is VSVACS-hard even if the game is invisible (this remains A^P-hard if the 
game is finite). All of our hardness results hold even if there are only two players and the 
game is symmetric. 
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2 Equilibria with certain properties in normal form 
games 

When one analyzes the strategic structure of a game, especialy from the viewpoint of a 
mechanism designer who tries to construct good rules for a game, finding a single equilibrium 
is far from satisfactory. More desirable equilibria may exist: in this case the game becomes 
more attractive, especially if one can coax the players into playing a desirable equilibrium. 
Also, less desirable equilibria may exist: in this case the game becomes less attractive. Before 
we can make a definite judgment about the quality of the game, we would like to know the 
answers to questions such as: What is the game's most desirable equilibrium? Is there a 
unique equilibrium? If not, how many equilibria are there? Algorithms that tackle these 
questions would be useful both to players and to the mechanism designer. 

Furthermore, algorithms that answer certain existence questions may pave the way to 
designing algorithms that construct a Nash equilibrium. For example, if we had an algorithm 
that told us whether there exists any equilibrium where a certain player plays a certain 
strategy, this could be useful in eliminating possibilities in the search for a Nash equilibrium. 

However, all the existence questions that we have investigated turn out to be AfV-hard. 
These are not the first results of this nature; most notably, Gilboa and Zemel provide some 
A/'P-hardness results in the same spirit [O] . We provide a single reduction which in demon- 



strates (sometimes stronger versions of) most of their hardness results, and interesting new 
results. Additionally, as we show in Section |3|, the reduction can be used to show j^V- 
hardness of counting the number of equilibria. 

To begin, we need some standard defintions from game theory. 

Definition 1 In a normal form game, we are given a set of agents A, and for each agent i, 
a strategy set and a utility function ui : Si x S 2 x . . . x T,\a\ — > 9ft. 



Definition 2 A mixed strategy <X; for player i is a probability distribution over Hi. A special 
case of a mixed strategy is a pure strategy, where all of the probability mass is on one element 
of lli. 



Definition 3 (Nash ||31|| ) Given a normal form game, a Nash equilibrium (NE) is vector 
of mixed strategies, one for each agent i, such that no agent has an incentive to deviate from 
its mixed strategy given that the others do not deviate. That is, for any % and any alternative 
mixed strategy o[, we have E[ui(sx, S2, ■ ■ ■ , Sj, • • • , sla|)] > E[ui(si, S2, ■ ■ ■ , s-, . . . , s\a\)], where 
each Si is drawn from o~i , and s • is drawn from o~[ . 

Now we are ready to present our reduction. 

Definition 4 Let <fi be a Boolean formula in conjunctive normal form. Let V be its set of 
variables (with \V\ = n), L the set of corresponding literals (a positive and a negative one 
for each variable)^, and C its set of clauses. The function v : L — * V gives the variable 

1 Thus, if x\ is a variable, x\ and —x\ are literals. We make a distinction between the variable x\ and 
the literal x\. 
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corresponding to a literal, e.g. v(xi) = v(—Xi) = x±. We define G(<f>) to be the following 
symmetric 2-player game in normal form. Let £ = £ 1 = £ 2 = LUl / UCU {/}. Let the 
utility functions be specified as follows: 

• Ul (l\ I 2 ) = u 2 {l 2 , 1 1 ) = 1 for all l\ I 2 eL with I 1 + -I 2 ; 

• U\(l, —I) = U2(—l, I) = —2 for all I G L; 

• Ui(l, x) = u 2 (x, I) = —2 for all I G L , x G £ — L; 

• Ui(v, I) = 112(1, v ) = 2 for all v G V , I G L with v(l) 7^ v; 

• Ui(v, I) = 112(1, v ) = 2 — n for all v G V , I G L with v(l) = v; 

• Ui(v, x) = U2(x, v) = —2 for all v G V , x G £ — L; 

• U\(c, I) = u 2 (l, c) = 2 for all c G C , I G L with I c; 

• Ui(c, I) — u 2 (l,c) = 2 — n for all c G C , I G L with I G c; 

• Ui(c, x) = u 2 (x, c) = —2 /or all c <E C , x <E T, — L; 

• «i(/,/)=«2(/,/)=0; 

• ui(f,x) = u 2 (x, f) = 1 /or allx eT,- {/}. 

Theorem 1 // l 2 , . . . , l n ) (where v(li) = Xi) satisfies <fi, then there is a Nash equilibrium 
of G ((f)) where both players play with probability ^, with expected utility 1 for each player. 
Furthermore, the only other Nash equilibrium is the one where both players play f , and 
receive expected utility each. 

Proof: We first demonstrate that these combinations of mixed strategies indeed do con- 
stitute Nash equilibria. If (I1J2, ■ ■ ■ , l n ) (where v(k) = Xi) satisfies and the other player 
plays U with probability ^, playing one of these U as well gives utility 1. On the other hand, 
playing the negation of one of these /j gives utility ^(—2) + B ^-(l) < 1. Playing some variable 
v gives utility ^(2 — n) + ^^-(2) = 1 (since one of the U that the other player sometimes 
plays has v(U) = v). Playing some clause c gives utility at most ^(2 — n) + ^^-(2) = 1 (since 
one of the U that the other player sometimes plays occurs in clause c, since the U satisfy 
0). Finally, playing / gives utility 1. It follows that playing any one of the U that the other 
player sometimes plays is an optimal response, and hence that both players playing each of 
these li with probability ^ is a Nash equilibrium. Clearly, both players playing / is also a 
Nash equilibrium since playing anything else when the other plays / gives utility —2. 

Now we demonstrate that there are no other Nash equilibria. If the other player always 
plays /, the unique best response is to also play / since playing anything else will give utility 
—2. Otherwise, given a mixed strategy for the other player, consider a player's expected 
utility given that the other player does not play /. (That is, the probability distribution 
over the other player's strategies is proportional to the probability distribution constituted 
by that player's mixed strategy, except / occurs with probability 0). If this expected utility 
is smaller than 1, the player is strictly better off playing / (which gives utility 1 when the 
other player does not play /, and also performs better than the original strategy when the 
other player does play /). So this cannot happen in a Nash equilibrium. 
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There are no Nash equilibria where one player always plays / but the other does not, 
so suppose both players play / with probability less than one. Consider the expected social 
welfare (E[u% +U2]), given that neither player plays /. It is easily verified that there is no 
outcome with social welfare greater than 2. Additionally, any outcome in which one player 
plays an element of V or C has social welfare strictly below 2. It follows that if either player 
ever plays an element of V or C, the expected social welfare given that neither player plays / 
is strictly below 2. By linearity of expectation it follows that the expected utility of at least 
one player is strictly below 1 given that neither player plays /, and by the above reasoning, 
this player would be strictly better off playing / instead of its randomization over strategies 
other than /. It follows that no element of V or C is ever played in a Nash equilibrium. 

So, we can assume both players only put positive probability on strategies in L U {/}. 
Then, if the other player puts positive probability on /, playing / is a strictly better response 
than any element of L (since both give utility 1 if the other player plays an element of L, 
but / does better if the other player plays /). It follows that the only equilibrium where / 
is ever played is the one where both players always play /. 

Now we can assume that both players only put positive probability on elements of L. 
Suppose that for some I G L, the probability that a given player plays either I or — I is less 
than -. Then the expected utility for the other player of playing v(l) is strictly greater than 
^(2 — n) + 21= i(2) = 1, and hence this cannot be a Nash equilibrium. So we can assume that 
for any / G L, the probability that a given player plays either I or —I is precisely -. 

If there is an element of L such that player 1 puts positive probability on it and player 
2 on its negation, both players have expected utility less than 1 and would be better off 
switching to /. So, in a Nash equilibrium, if player 1 plays I with some probability, player 
2 must play I with probability -, and thus player 1 must play I with probability -. Thus 
we can assume that for each variable, exactly one of its corresponding literals is played with 
probability - by both players. It follows that in any Nash equilibrium (besides the one where 
both players play /) , literals that are sometimes played indeed correspond to an assignment 
to the variables. 

All that is left to show is that if this assignment does not satisfy <f), we do not have a 
Nash equilibrium. Let c G C be a clause that is not satisfied by the assignment, that is, none 
of its literals are ever played. Then playing c would give utility 2, and both players would 
be better off playing this. ■ 

Hence, there exists a Nash equilibrium in G(4>) where each player gets utility 1 if and only 
if is satisfiable; otherwise, the only equilibrium is the one where both players play / and 
each of them gets 0. Since any sensible definition of welfare optimization would prefer the 
first kind of equilibrium, it follows that determining whether a "good" equilibrium exists is 
hard for any such definition. Additionally, the first kind of equilibrium is, in various senses, 
an optimal outcome for the game, even if the players were to cooperate, so even finding out 
whether such an optimal equilibrium exists is hard. The following corollaries illustrate these 
points (each corollary is immediate from Theorem |IJ). 

Corollary 1 Even in symmetric 2-player games, it is AfV-hard to determine whether there 
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exists a NE with expected (standard) social welfare (E[ u i\) a t least k, even when k is 

l<i<\A\ 

the maximum social welfare that could be obtained in the game. 

Corollary 2 Even in symmetric 2-player games, it is MV-hard to determine whether there 
exists a NE where all players have expected utility at least k, even when k is the largest 
number such that there exists a distribution over outcomes of the game such that all players 
have expected utility at least k. 

Corollary 3 Even in symmetric 2-player games, it is MV-hard to determine whether there 
exists a Pareto-optimal NE. (A distribution over outcomes is Pareto- optimal if there is no 
other distribution over outcomes such that every player has at least equal expected utility, 
and at least one player has strictly greater expected utility). 

Corollary 4 Even in symmetric 2-player games, it is MV-hard to determine whether there 
exists a NE where player 1 has expected utility at least k. 

Some additional interesting corollaries are: 

Corollary 5 Even in symmetric 2-player games, it is MV-hard to determine whether there 
is more than one Nash equilibrium. 

Corollary 6 Even in symmetric 2-player games, it is MV-hard to determine whether there 
is an equilibrium where player 1 sometimes plays x G Si. 

Corollary 7 Even in symmetric 2-player games, it is MV-hard to determine whether there 
is an equilibrium where player 1 never plays x 6 Si. 

All of these results indicate that it is hard to obtain summary information about a game's 
Nash equilibria. (Corollary [| and weaker versions of Corollaries |2|, |6| and |^ were first proven 
by Gilboa and Zemel flTB|.) 

3 Counting the number of equilibria in normal form 
games 

Existence questions do not tell the whole story. In general, we are interested in character- 
izing all the equilibria of a game. One rather weak such characterization is the number of 
equilibria^. We can use Theorem [I| to show that even determining this number in a given 
normal form game is hard. 

2 The number of equilibria in normal form games has been studied both in the worst case ]3C| | and in the 
average case [p9] . 
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Corollary 8 Even in symmetric 2-player games, counting the number of Nash equilibria is 
#V-hard. 



Proof: The number of Nash equilibria in our game G(4>) is the number of satisfying assign- 
ments to the variables of 0, plus one. Counting the number of satisfying assignments to a 
CNF formula is #P-hard pf. ■ 



It is easy to construct games where there is a continuum of Nash equilibria. In such games, 
it would be more meaningful to ask how many distinct continuums of equilibria there are. 
More formally, one can ask how many maximal connected sets of equilibria a game has (a 
maximal connected set is a connected set which is not a proper subset of a connected set). 

Corollary 9 Even in symmetric 2-player games, counting the number of maximal connected 
sets of Nash equilibria is j^V-hard. 

Proof: Every Nash equilibrium in G(<f>) constitutes a maximal connected set by itself, so the 
number of maximal connected sets is the number of satisfying assignments to the variables 
of 0, plus one. ■ 

The most interesting ^P-hardness results are the ones where the corresponding existence and 
search questions are easy, such as counting the number of perfect bipartite matchings. In the 
case of Nash equilibria, the existence question is completely trivial: it has been analytically 
shown (by Kakutani's fixed point theorem) that a Nash equilibrium always exists |31| . The 
complexity of the search question remains open. 



4 Pure-strategy Bayes-Nash equilibria 

Equilibria in pure strategies are particularly desirable because they avoid the uncomfortable 



requirement that players randomize over strategies among which they are indifferent |L2 
In normal form games with small numbers of players, it is easy to determine the existence 
of pure-strategy equilibria: one can simply check, for each combination of pure strategies, 
whether it constitutes a Nash equilibrium. However, this is not feasible in Bayesian games, 
where the players have private information about their own preferences (represented by 
types). Here, players may condition their actions on their types, so the strategy space of 
each player is exponential in the number of types. 

In this section, we show that the question of whether a pure-strategy Bayes-Nash equi- 
librium exists is in fact A/"P-hard even in symmetric two-player games. First, we need the 
standard definition of a Bayesian game and Bayes-Nash equilibrium from game theory. 

Definition 5 In a Bayesian game, we are given a set of agents A; for each agent i, a set of 
types 0j/ a commonly known prior distribution <fi over Bi x 02 x . . . x 0|a|/ for each agent i, a 
set of strategies Sj/ and for each agent i, a utility function «j : 9, x Si x E2 x . . . x E141 — > 9ft. 
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Definition 6 (Harsanyi [|15|| ) Given a Bayesian game, a Bayes-Nash equilibrium (BNE) 
is a vector of mixed strategies, one for each pair i,9i G ©», such that no agent has an 
incentive to deviate, for any of its types, given that the others do not deviate. That is, for 
any i,9i G ©i, and any alternative mixed strategy we have 

^e-i\ei[E[ui{9^ s^ 01 , s 2fi2 , . . . , s ifil , s\ A \^ A] )]] > E e _ t \e t [E[u i (9 il s lfil , s 2 ,e 2 , . . . , s' id ., s\ A \^ A] )]] 

where each is drawn from <Ji$ v and s' i6 . is drawn from (T^g . 

We can now define the computational problem that we study. 

Definition 7 (PURE-STRATEGY-BNE) We are given a Bayesian game. We are asked 
whether there exists a BNE where all the strategies a^Q i are pure. 

To show our A/'P-hardness result, we will reduce from the SET-COVER problem. 

Definition 8 (SET-COVER) We are given a set S = {sj, . . . , s n }, subsets Si, S 2 , ... , S m 

of S with Ui<i< m Si = S, and an integer k. We are asked whether there exist S Cl , S C2 , . . . , S Ck 
such that Ui<i<fc S Cl = S. 



Theorem 2 PURE-STRATEGY-BNE is AfV '-hard, even in symmetric 2-player games where 
cf) is uniform. 

Proof: We reduce an arbitrary SET-COVER instance to the following PURE-STRATEGY- 
BNE instance. Let there be two players, with O = ©i = ©2 = {0 l , . . . , 8 h }. is uniform. 
Furthermore, E = £1 = £ 2 = {Si, S 2 , . . . , S m , si, s 2 , . . . , s n }. The utility functions we choose 
in fact do not depend on the types, so we omit the type argument in their definitions. They 
are as follows: 

• Ui(Si, Sj) = u 2 (Sj, Si) = 1 for all Si and Sf 

• Ui(Si, Sj) = u 2 {sj, Si) = 1 for all Si and Sj ^ S^ 

• Ui(Si, Sj) = u 2 (sj, Si) = 2 for all Si and Sj G Sf, 

• Ui(si, Sj) = u 2 (sj, Si) = —3k for all s« and Sj] 

• Ui(sj, Si) = u 2 (Si, Sj) = 3 for all Si and Sj ^ Sf, 

• Ui(sj, Si) = u 2 (Si, Sj) = —3k for all Si and Sj G S^ 

We now show the two instances are equivalent. First suppose there exist S Cl , S C2 , . . . , S Ck 
such that Ui<i<fc Sa — S. Suppose both players play as follows: when their type is 6^, they 
play S Ci . We claim that this is a BNE. For suppose the other player employs this strategy. 
Then, because for any Sj, there is at least one S Ci such that sj G S Ci , we have that the 
expected utility of playing Sj is at most |(— 3k) + ^rr3 < 0. It follows that playing any of 
the Sj (which gives utility 1) is optimal. So there is a pure-strategy BNE. 

On the other hand, suppose that there is a pure-strategy BNE. We first observe that in 
no pure-strategy BNE, both players play some element of S for some type: for if the other 
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player sometimes plays some Sj, the utility of playing some Sj is at most r(— 3k) + ^-3 < 0, 
whereas playing some Si instead guarantees a utility of at least 1. So there is at least one 
player who never plays any element of S. Now suppose the other player sometimes plays 
some Sj. We know there is some Si such that Sj & Si. If the former player plays this Si, 
this will give it a utility of at least j2 + ^-1 — 1 + \ - Since it must do at least this well 
in the equilibrium, and it never plays elements of S, it must sometimes receive utility 2. It 
follows that there exist S a and s b G S a such that the former player sometimes plays S a and 
the latter sometimes plays Sb- But then, playing Sb gives the latter player a utility of at most 
r(— 3k) + ^£-3 < 0, and it would be better off playing some Si instead. (Contradiction.) It 
follows that in no pure-strategy BNE, any element of S is ever played. 

Now, in our given pure-strategy equilibrium, consider the set of all the Si that are played 
by player 1 for some type. Clearly there can be at most k such sets. We claim they cover S. 
For if they do not cover some element Sj, the expected utility of playing Sj for player 2 is 3 
(because player 1 never plays any element of 5*). But this means that player 2 (who never 
plays any element of 5* either) is not playing optimally. (Contradiction.) Hence, there exists 
a set cover. ■ 



If one allows for general mixed strategies, a Bayes-Nash equilibrium always exists [[12 . 
However, the question of how efficiently one can be constructed remains open. 

5 Pure-strategy Nash equilibria in stochastic (Markov) 
games 

We now shift our attention from single-shot games to games with multiple stages. In each 
stage, the players get to act and obtain payoffs. There has already been some research into 
the complexity of playing repeated and sequential games. For example, determining whether 
a particular automaton is a best response is AA'P-complete || ; it is TVP-complete to compute 
a best-response automaton when the automata under consideration are bounded f3"Tfl ; the 



question of whether a given player with imperfect recall can guarantee itself a given payoff 
using pure strategies is AfP-complete pD[ ; and in general, best-responding to an arbitrary 



strategy can even be noncomputable [[51]] . In this section, we present, to our knowledge, the 
first VSVACS-h&rdiaess result on the existence of a pure-strategy equilibrium. 

A multi-stage game is typically represented as a stochastic (Markov) game, where there is 



an underlying set of states, and the game shifts between these states from stage to stage J|5 
[46], |P2"|| . At every stage, each player's payoff depends not only on the players' actions, but also 
on the state. Furthermore, the probability of transitioning to a given state is determined by 
the current state and the players' current actions. Hardness results for such games cannot be 
obtained simply by formulating a known hard game such as generalized Go |26| or QSAT ||8 



as a Markov game, because such a formulation would have to specify an exponential number 
of states. Even if the number of states is polynomial, one might suspect hardness due to the 
fact that the strategy spaces are extremely rich. However, in this section we show VSVACS- 
hardness even in a variant where the strategy spaces are simple (in the sense that the players 



S 



cannot condition their actions on events in the game). 

Definition 9 A stochastic (Markov) game consists of the following. 

• A set of players A; 

• A set of states S , among which the game transits; 

• For each player i, a set of actions £j that can be played in any state; 

• A transition probability function p : SxT,i x . . .xS^i xS — ► [0,1], where p(si, a 1 , . . . , a n , s 2 ) 
gives the probability of the game being in state S2 in the next stage given that the current 
state of the game is S\ and the players play actions CL\ , . . . , a n ; 

• For each player i, a payoff function Ui : S x S x x . . . H\a\ - ► 9ft, where Ui(s, a±, . . . , au|) 
gives the payoff to player i in state s where the players play actions ai, . . . , a n ; 

oo 

• A discount factor 5 such that the total utility of agent i is J2 S k Ui(s k , a k , . . . , cl\m), 

k=0 

where s k is the state of the game at stage k and the players play actions a k , . . . , a„ in 
stage k. 

In general, a player need not always be aware of the current state of the game, the actions 
the others played in previous stages, or the payoffs that the player has accumulated. In the 
extreme case, players never find out any of these and are hence playing blindly. We call 
such a Markov game invisible. It is relatively easy to specify a pure strategy in an invisible 
Markov game, because there is nothing to condition on. Hence, such a strategy is "simply" 
an infinite sequence of actions (for player i, a sequence {a k }, where it plays action a k in stage 
k, regardless) .0 In spite of this apparent simplicity of the game, we show that determining 
whether pure-strategy equilibria exist is extremely hard. 

Definition 10 (PURE-STRATEGY-INVISIBLE-MARKOV-NE) We are given an 
invisible Markov game. We are asked whether there exists a Nash equilibrium where all the 
strategies are pure. 

We show that this problem is VSVACS-ha.rd, by reducing from PERIODIC-SAT, which 
is known to be VSVACS-complete f36j. 

Definition 11 (PERIODIC-SAT) We are given a CNF formula 0(0) over the variables 
{xi . . . x° } U {x\ . . . x^}. Let 4>(k) be the same formula, except that all the superscripts are 
incremented by k. We are asked whether there exists a Boolean assignment to the variables 
Ufc=o,i,...{ x i • • • x n} su °h ^at <p{k) is satisfied for every k = 0, 1, 



Theorem 3 P URE-STRA TEG Y-INVISIBLE-MARKO V-NE is VSVACS-hard, even when 
the game is symmetric, 2-player, and the transition process is deterministic. 



3 We do not need to worry about issues of credible threats and subgame perfection in this setting, so we 



can simply use Nash equilibrium as our solution concept 28 
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Proof: We reduce an arbitrary PERIODIC-SAT instance to the following symmetric 2- 
player PURE-STRATEGY-INVISIBLE- MARKOV-NE instance. The state space is S = 
{si}i<i< n U 0- )C }i<i<2n;ceC U {i 2 c }i<i<2n;cec U {r}, where C is the set of clauses in 0(0). 
Furthermore, E = Si = £2 = {£, /} U C. The transition probabilities are as follows: 

• p(si,£\z 2 ,Sj+i(mocin)) = 1 for 1 < i < n and all x\x 2 G E; 

• p(si, ft 1 , b 2 , s 2 ) = 1 for all ft 1 , b 2 G {f , /}; 

• p(si, c, 6, ^ c ) = 1 for all 6 G {t, /} and c G C; 

• 6, c, t| c ) = 1 for all b G {i, /} and c G C; 

• p(si, c 1 , c 2 , r) = 1 for all c 1 , c 2 G C; 

• pC^o^j^^+i.c) = 1 for all 1 < i < 2n, j G {1,2}, c G C, and x*,x 2 G £; 

• p(*2n,o z 1 , r ) = 1 f° r ah j £ {1, 2}, c G C, and x 1 , x 2 G E; 

• p(r, x 1 , x 2 , r) = 1 for all x*,x 2 G £. 

Some of the utilities obtained in a given stage are as follows (we do not specify utilities 
irrelevant to our analysis): 

• Ui(si, x 1 , x 2 ) = U2(si, x 2 , x 1 ) = for 1 < % < n and all x 1 , x 2 G E; 

• Ul (si, b\ b 2 ) = u 2 (s 1 , b 2 , b 1 ) = for all b 1 ,b 2 G {t, /}; 

• Ui(si, c, 6) = «2(si, 6, c) = 1 for all b G {t, /} and c G C, when setting variable re? to b 
does not satisfy c; 

• Mi(si, c, 6) = u 2 (si, b, c) = —1 for all b G {t, /} and c G C, when setting variable to 
b does satisfy c; 

• Mi(si, c 1 , c 2 ) = u 2 (si, c 2 , c 1 ) = — 1 for all c 1 , c 2 G C; 

• Wi(*fc n+i)C , x, b) = u 2 (t 2 kn+i c , b, x) = for fc G {0, 1}, 1 < i < n, all c G C and 6 G {t, /} 
such that setting variable xf to 6 does not satisfy c, and all x G S; 

• Wi(*L-H ) o a; > 6 ) = Mtln+i^^x) = - 4 for ^ e {0,1}, 1 < « < n, all c G C and 
& G {t, /} such that setting variable x\ to b does satisfy c, and all x G S; 

• Wi(*fcn-H ) o a: > c ') = M 2(tL+i, c ,c',x) = for /c G {0, 1}, 1 < i < n, all c, c' G C, and all 

ieE. 

Additionally, the game played in state r is some symmetric zero-sum game without a 
pure-strategy equilibrium (for example, a generalization of rock-paper-scissors) with very 

small payoffs. Finally, the discount factor is 5 — (|) 271+1 (so that 5 2n > \)- 

We start our analysis with a few observations. First, there can be no pure-strategy 
equilibrium in which state r is reached at some point, because (since r is an absorbing state) 
this would require that some pure-strategy equilibrium of the game in state r were played 
whenever state r occurred. (Otherwise a player who is not best-responding in one of these 
stages could simply switch to a best response in this stage, and because the game is invisible, 
the rest of the game would remain unaffected, so this would give higher utility.) But such an 
equilibrium does not exist. Second, if we ever reach one of the t\ c states, we will inevitably 
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reach state r at some point after this. It follows that all pure-strategy Nash equilibria never 
leave the states. 

Now suppose an assignment satisfying the periodic SAT formula exists. Let both players 
play as follows: in stage kn + i (with 1 < i < n), b G {t, /} is played, where b is the value 
that the variable x k is set to. Clearly, both players receive utility with these strategies. 
Does either player have an incentive to deviate? The only deviation of any significance is 
to play some c G C when the current state is S\. So, without loss of generality (because 
of the symmetry of the game), say player 2 deviates to playing c G C in stage kn + 1 
(when the state is si). We know that in the satisfying assignment, some variable x\ among 
x k , . . . , x*, x k+1 , . . . , x^ +1 is set to some b such that setting x l ~ k to b satisfies c. If it is x\, 
which is set to b, then in stage kn + 1 player 1 plays b, and player 2 gets payoff —1 in 
this stage since we are in state si and setting x\ to b satisfies c. Otherwise, if it is x\ with 
! = fc+ 1 or i ^ 1, which is set to b, then player 2 will get payoff 1 in stage kn + 1, but in 
stage In + i player 1 plays b, and player 2 gets payoff —4 in this stage since we are in state 
tfi-k)n+i,c an d setting x\~ k to b satisfies c. The discounting is insignificant enough that this 
more than cancels out the 1 earned in stage kn + 1. Player 2 will get (at most) in the other 
stages up to the first stage in state r, and given that we made the payoffs in the game in 
state r sufficiently small relative to 5, player 2 will not earn enough in the remaining stages 
to cancel out its losses so far. So there is no incentive to deviate. Thus, a pure-strategy NE 
exists. 

On the other hand, suppose that no assignment satisfying the periodic SAT formula ex- 
ists. Let us investigate whether a Nash equilibrium could exist. We know that in such a 
Nash equilibrium we never leave the Sj, so both players receive utility 0, and no c is ever 
played in a stage with state si. Since playing a c in one of the other stages can have no 
deterrent value, we may suppose that only elements of {t, /} are played. Now consider the 
following assignment to the x\ : if player 1 plays b in stage kn + i, x k is set to b. Since no 
assignment satisfying the periodic SAT formula exists, we know there is some clause c and 
some k such that no variable x\ among x k , . . . , x*, x k+1 , . . . , x^ 1 is set to some b such that 
setting x l ~ k to b satisfies c. But then, if player 2 deviates to play this c in stage kn + 1, it will 
receive payoff 1 in this stage, and payoff in all the remaining stages up to the first stage in 
state r. Furthermore, player 2 can guarantee itself at least payoff in each stage in state r, 
as this state corresponds to a zero-sum symmetric game. It follows that this deviation gives 
player 2 positive utility and is hence beneficial. Thus, no pure-strategy NE exists. ■ 

A simpler version of the same argument shows a weaker form of hardness for the case 
where the game is restricted to have only finitely many stages (we omit the proof due to 
limited space): 

Theorem 4 PURE-STRATEGY-INVISIBLE-MARKOV-NE is MV-hard, even when the 
game is symmetric, 2-player, the transition process is deterministic, and the number of stages 
in the game is finite. 
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6 Conclusions and future research 



Noncooperative game theory provides a normative framework for analyzing strategic inter- 
actions. However, for the toolbox to be operational, the solutions it defines will have to be 
computed. 

In this paper, we provided a single reduction that 1) demonstrates A^P-hardness of deter- 
mining whether Nash equilibria with certain natural properties exist, and 2) demonstrates 
the T^P-hardness of counting Nash equilibria (or connected sets of Nash equilibria). We 
also showed that 3) determining whether a pure-strategy Bayes-Nash equilibrium exists is 
A/'P-hard, and that 4) determining whether a pure-strategy Nash equilibrium exists in a 
stochastic (Markov) game is VSV AC S-ha,rd even in invisible games (and remains AAP-hard 
if the game is finite). All of our hardness results hold even if there are only two players and 
the game is symmetric. 

The intersection of computer science and economics is a fertile research area, and has 
many exciting avenues for future research. For example, within the area of computational 
complexity, there are open questions regarding the complexity of executing various mech- 



anism optimally (e.g., J|, |42|, |16| ) or approximately (e.g. |3J, P5|), the complexity of 



manipulating various mechanisms (e.g., B 0, 0, |44l]), the complexity of designing mech- 



anisms (that lead to desirable outcomes) [40fl , and the complexity of deciding what 



information to elicit from the players in various mechanisms jnj. Another avenue involves 
studying more sophisticated equilibrium notions which take into account that players have 
limited memory (e.g. [O, [I], [TJ|, [5DJ) or limited capability to solve optimization problems 



(e.g. |23], [24], [Hj, |34[]). There are also open issues on communication complexity in games 
(e.g., [[J^, 0, |S], [IT], |SD[), and on the complexity of computing general equilibria ("market 
equilibria") (e.g., IfjCfl ) and other solutions. 

There are numerous open research questions even in the area of computing solutions to 
noncooperative games. Some recent work has focused on novel knowledge representations 
which, in certain settings, can drastically speed up equilibrium finding (e.g. [[H], ^2fl). One 
avenue of future theoretical work includes identifying restricted classes of games for which 
equilibria (or equilibria with certain properties) can be found fast. Another avenue involves 
studying the complexity of characterizing (some of) the equilibria of a game partially. Yet 
another avenue includes analyzing the computational complexity of other solution concepts 
from noncooperative game theory. 
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